Llama-4-Maverick-17B-128E-Instruct-NVFP4 is a multilingual large language model that has undergone FP4 quantization processing. It is based on the Meta-Llama-3.1 architecture and is specifically designed for business and research purposes. By quantizing the weights and activations to the FP4 data type, this model significantly reduces the disk space and GPU memory requirements while maintaining good performance.

Natural Language ProcessingSafetensorsMultiple Languages

This is the INT8 quantized version of the Devstral-Small-2507 model. By quantizing the weights and activation values to 8-bit integers, it significantly reduces the GPU memory and disk space requirements by approximately 50% while maintaining good performance on coding tasks.

Natural Language ProcessingSafetensorsEnglish

The FP8 quantized version of the SmolLM3-3B model developed by RedHatAI. By quantizing weights and activation values to the FP8 data type, it significantly reduces GPU memory requirements and disk space usage while improving computational throughput.

Natural Language ProcessingSafetensorsMultiple Languages

The NVIDIA Qwen3-235B-A22B FP8 model is a quantized version of the Alibaba Cloud Qwen3-235B-A22B model. It adopts an optimized Transformer architecture and is an autoregressive language model. This model significantly reduces disk space and GPU memory requirements through FP8 quantization technology while maintaining high inference accuracy, making it suitable for various AI application scenarios.

Natural Language ProcessingSafetensors

A large language model with 17B parameters based on the Llama-4 architecture. Optimized through INT4 weight quantization technology, it reduces the GPU memory and disk space requirements by 75%. It supports multilingual text generation tasks and can be efficiently deployed on the vLLM inference framework.

MultimodalSafetensorsMultiple Languages

The DeepSeek-R1-0528 model after quantization processing significantly reduces the requirements for GPU memory and disk space by quantizing the weights to the INT4 data type.

Natural Language ProcessingSafetensors

The INT4 quantized version of Qwen3-8B. The weights are quantized to 4-bit integers through the GPTQ algorithm, significantly reducing the disk space and GPU memory requirements by 75% while maintaining performance close to the original model. It is suitable for scenarios such as inference, function calls, and multilingual instruction following.

Natural Language ProcessingTransformers

This model is the FP8 quantized version of Bielik-4.5B-v3.0-Instruct, utilizing AutoFP8 technology to quantize weights and activations into FP8 data type, reducing approximately 50% of disk space and GPU memory requirements.

Natural Language ProcessingGgufOther

This is an FP8 dynamic quantization version based on the Bielik-1.5B-v3.0-Instruct model, adapted for vLLM or SGLang inference frameworks. It uses AutoFP8 quantization technology to reduce parameter bytes from 16-bit to 8-bit, significantly lowering disk space and GPU VRAM requirements.

Natural Language ProcessingGgufOther

Qwen3-8B-FP8-dynamic is an optimized version of the Qwen3-8B model through FP8 quantization, significantly reducing GPU memory requirements and disk space usage while maintaining the original model's performance.

Natural Language ProcessingTransformers

INT4 weight-quantized version of DeepSeek-R1, reducing GPU memory and disk space requirements by approximately 50% while maintaining original model performance.

Natural Language ProcessingSafetensors

This model is an INT4 quantized version of google/gemma-3-12b-it, using the GPTQ algorithm to reduce parameters from 16-bit to 4-bit, significantly decreasing disk space and GPU memory requirements.

MultimodalTransformers

This is a quantized version of DeepSeek-R1-Distill-Qwen-1.5B. By quantizing the weights and activations to the INT8 data type, it effectively reduces the GPU memory requirement and disk space usage while improving the computational throughput. This model is developed by Neural Magic and is based on the Qwen2ForCausalLM architecture.

Natural Language ProcessingTransformers

This is the INT4 quantized version of DeepSeek-R1-Distill-Qwen-32B. The model weights are quantized to the INT4 data type through the GPTQ algorithm, significantly reducing the disk space and GPU memory requirements by 75% while maintaining performance close to that of the original model.

Natural Language ProcessingTransformers

This is a quantized version of Meta-Llama-3.1-70B-Instruct. It reduces GPU memory requirements and disk space through INT8 weight quantization while improving computational throughput. It is suitable for multilingual business and research scenarios.

Natural Language ProcessingTransformersMultiple Languages

A quantized version of Meta-Llama-3.1-8B-Instruct, optimized to reduce disk space and GPU memory requirements, suitable for chat assistant scenarios in English business and research.

Natural Language ProcessingTransformersMultiple Languages

This is a quantized version of Meta-Llama-3.1-70B-Instruct. By performing FP8 quantization on weights and activations, it reduces approximately 50% of disk space and GPU memory requirements. It supports multiple languages, is suitable for business and research purposes, and can use the vLLM backend for efficient inference.

Natural Language ProcessingTransformersMultiple Languages

Meta-Llama-3-8B-Instruct-FP8 is a quantized version of Meta-Llama-3-8B-Instruct. By quantizing weights and activations to the FP8 data type, it maintains high performance while reducing disk space and GPU memory requirements. It is suitable for English commercial and research purposes and can be used in chat scenarios similar to assistants.

Natural Language ProcessingTransformersEnglish